Evaluation of Cognitive levels and Item writing flaws in Medical Pharmacology Internal Assessment Examinations
نویسندگان
چکیده
OBJECTIVES This study aimed to evaluate the cognitive levels of Multiple Choice Questions (MCQs) & Short Answer Questions (SAQs) and types of Item Writing Flaws (IWFs) in MCQs in Medical Pharmacology internal assessment exams. METHODS This descriptive, study was conducted over a period of six months, from December 2015 to May 2016 and evaluated six internal assessment examinations comprising SAQs and MCQs. A total of 150 MCQs and 43 SAQs were analyzed. These questions were administered to third-year medical students in the year of 2015. All SAQs were reviewed for their cognitive levels and MCQs were reviewed for cognitive levels as well as for IWFs. Items were classified as flawed if they contained one or more than one flaw. The cognitive level of the questions was determined by the modified Bloom's taxonomy. RESULTS The proportion of flawed items out of 150 items in six exams ranged from 16% to 52%. While the percentage of total flawed items was 28%. Most common types of flaws were implausible distractors 19.69% (26), extra detail in correct option 18.18% (24), vague terms 9.85% (13), unfocused stem 9.09% (12) and absolute terms 9.09% (12). The two-third of MCQs 97(64.67%) were assessing the recall of information, while 29 (19.33%) and 24 (16%) were assessing the interpretation of data and problem-solving skills respectively. The majority of the SAQs (90.7%) were assessing recall of the information and only 9.3% were assessing interpretation of data while none of the questions was assessing the problem-solving skills. CONCLUSIONS The cognitive level of assessment tools (SAQs & MCQs) is low, and IWFS are common in the MCQs. Therefore, faculty should be urged and groomed to design problem-solving questions which are devoid of any flaws.
منابع مشابه
The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.
Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and ...
متن کاملThe impact of item-writing flaws and item complexity on examination item difficulty and discrimination value
BACKGROUND Failure to adhere to standard item-writing guidelines may render examination questions easier or more difficult than intended. Item complexity describes the cognitive skill level required to obtain a correct answer. Higher cognitive examination items promote critical thinking and are recommended to prepare students for clinical training. This study evaluated faculty-authored examinat...
متن کاملReviewing the results of qualitative and quantitative analysis of MCQs in Introduction to clinical medicine course
Background: At most of the medical universities, MCQ-based examinations are often use as the first component, particularly to ensure the candidates have an adequate knowledge base, prior to entering subsequent clinical examinations. Method: In ICM , 3973 MCQs were collected from tests over a four-year period from 2005 to2009. Questions were evaluated for 10 frequently occurring item writing f...
متن کاملEvaluation of Multiple Choice and Short Essay Question items in Basic Medical Sciences
OBJECTIVES To evaluate Multiple Choice and Short Essay Question items in Basic Medical Sciences by determining item writing flaws (IWFs) of MCQs along with cognitive level of each item in both methods. METHODS This analytical study evaluated the quality of the assessment tools used for the first batch in a newly established medical college in Karachi, Pakistan. First and sixth module assessme...
متن کاملEducation techniques for lifelong learning: writing multiple-choice questions for continuing medical education activities and self-assessment modules.
The multiple-choice question (MCQ) is the most commonly used type of test item in radiologic graduate medical and continuing medical education examinations. Now that radiologists are participating in the maintenance of certification process, there is an increased need for self-assessment modules that include MCQs and persons with test item-writing skills to develop such modules. Although princi...
متن کامل